1. Field of the Invention
The present invention relates to a signal processing device and a signal processing method, and a program and recording medium thereof, and particularly relates to an image-taking device, a signal processing device and a signal processing method, and a program and recording medium thereof whereby, for example, suitable image signals are obtained for signal processing of images signals, and the image signals are subjected to signal processing, thereby yielding high-quality image signals.
2. Description of the Related Art
Image-taking devices, an example of which is a digital camera (either still or video), have sensor means (or imaging means) such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) imager (also known as a “CMOS sensor”), for example, which receive subject light (light from a subject) and output image signals corresponding to the amount of light received. The sensor means act as a sensor for outputting image signals corresponding to the subject light in the form of electric signals by sensing the light from the subject and performing photoelectric conversion thereof.
There are several ways of using the sensor means (imaging means), such as the single-sensor system and the three-sensor system, for example. With the single-sensor system, color filters transmitting R (Red), G (Green), and B (Blue) light, e.g., optical filters arrayed in a pattern called a Bayer array, and one of the R. G, or B light is cast into the sensor pixels. Accordingly, each pixel of the sensor receives light of one of R, G, or B, and an image signal having the signal component of one of R, G, or B is output for one pixel. Thus, with the single-sensor system, each pixel making up the image obtained from the sensor have only one signal component of R, G, or B, so interpolation is performed later for signal components which each pixel do not have. For example, taking notice of a pixel containing only an R signal (component), the G signal and B signal of the pixel of interest are predicted by a nearby pixel having only a G signal and a nearby pixel having only a B signal (see Japanese Unexamined Patent Application Publication No. 2002-135797, for example).
Also, there is signal processing called demosaic wherein pixels having only one of the R, G, or B signals obtained from a single CCD are converted into pixels having all of the R, G, and B signals (see International Publication No. 00/49812, for example).
On the other hand, with a three-sensor system, the sensor means are configured of three sensors, one each for R, G, and B, so that the R sensor receives the R light, the G sensor receives the G light, and the B sensor receives the B light, such that image signals having the three components of the R signal, G signal, and B signal for each pixel are output.
Now, with image-taking devices with a three-sensor system, taking notice of a certain light ray, the light ray of interest is split into the R, G, and B light, by a prism, so that the R sensor receives the R light, the G sensor receives the G light, and the B sensor receives the B light. Accordingly, each of the sensors for R, G, and B are placed at optically equivalent (same) positions so that the R, G, and B light of the ray of interest is received at the same position on each sensor for each of R, G, and B. Note that an image-taking device has also been proposed wherein the G sensor is positioned so as to be offset by ½ pixel as to the R sensor and B sensor in order to obtain image signals with improved resolution in the vertical direction (see Japanese Unexamined Patent Application Publication No. 08-256345, for example).
Also, with conventional devices, the output of the sensing means such as the Charge Coupled Device (hereafter also referred to simply as “CCD”) or Complementary Metal Oxide Semiconductor (CMOS) imager (hereafter also referred to simply as “CMOS sensor”, or “CMOS”) is amplified to obtain image signals of a suitable level.
Amplification of the output of sensor means is carried out by an amplifier such as an AGC (Auto Gain Controller). However, in the event that the contrast of the subject is great, there may be cases wherein obtaining suitable image signals level with a single amplifier is difficult. Accordingly, Japanese Unexamined Patent Application Publication No. 06-086155 discloses a method for easily amplifying a wide dynamic range by amplifying the output of the sensor means with two amplifiers each having different gain.
Now, downstream of the sensor means, signal processing is often performed in order to improve the image quality of the image signals output by the sensor means. That is to say, the CCD or CMOS sensor of the sensor means (imaging means) receive incident light (subject light within a range corresponding to the pixels over a predetermined exposure time, and output image signals corresponding to the amount of light received. Accordingly, it may be said that the sensor performs sampling, as if it were, of the time-wise and space-wise continuous light within a range corresponding to the pixels over a predetermined exposure time, and outputs the sampling results as image signals (pixel values).
In this way, the image signals output by the sensor means are the sampling results of the time-wise and space-wise continuous light, meaning that a part of the information contained in the original light is missing. Accordingly, the image signals output by the sensor means are deteriorated with regard to image quality (i.e., the amount of information) as compared with the original light. This is why signal processing is performed downstream from the sensor means, to raise the image quality of the image signals of which the image quality has deteriorated as compared with the original light.
Now, with conventional arrangements, the sensor means are manufactured giving no regard whatsoever to the signal processing performed downstream therefrom, and operate to output image signals giving no regard whatsoever to the signal processing performed downstream therefrom. Accordingly, the sensor means have properties which are unrelated to the signal processing performed downstream, i.e., the sensor means take into consideration none of the signal processing performed downstream. Thus, not only do the sensor means have properties which are unrelated to the signal processing performed downstream, but also operate in a uniform manner to output image signals, meaning that there is a limit to how far the image quality can be improved by the signal processing performed downstream from the sensor means.
Conversely, if image signals suitable for signal processing performed downstream were to be output at the sensor means, image signals with further-improved image quality could be obtained by that signal processing.